From Schemes to Validation

UID/LIN/03213/2013Formal analyses of human movement are intended to be - by nature - objective. Despite the great interest in Gesture Studies of descibing gesture and other body movement forms, there is no one standardized guideline for the formal transcription nor annotation of body movementes. Thus, scienctists are left to adapt versions of pre-existing annotation schemes or develop their own. This paper aims at providing an overview of the gamut of annotation schemes used in the multimodal communication literatura in order to raise questions about how researchers define, treat and analyze body movements in their data. Differences of definitions cause problems when it comes to comparing research findings and are directly connected to the question of how body movement units are identified and classified by the research community. On the one hand, there is the problem of formal and functional labeling that are often collapsed in the adopted annotation scheme; on the other, definitional diversity affects humans raters' evaluation and judgmnt, not to mention differences in the annotation process when marking start- and end-points of a movement unit. Another issue is that human movement is often studied in a fragmented manner, where researchers (without blame) focus only on certain articulatores and not others. Whereas manual gestures are most studied within the the fiel, and some attempts have been undertaken for its standardization (inter alia Bressem, Ladewig, & Müller 2013; Lausberg & Sloetjes 2009), a structured annotation guideline for other articulators' expressions has yet to be reached (cf. "head-gestures annotation schemes": Kousidis 2013; Poggy 2010; Heylen 2008; Cerrato 20017; Allwood & Cerrato 2003). Consequently, a comprehensive annotation scheme containing all body articulators is unavailable to gesture researchers, although within the performing arts domain we find an example of that type (i.e. Laban movement analyses and notation). Besides the problems of definig the movement units and the segmentation issues, researchers also face obstacles in processing the data and the estimation of their reliability and validity. Already the widely used statistical coefficients for the measurement of inter-rater agreement (i.e. Fleiss' kappa, Krippendorff's alfa, Cohen's kappa), are problematic for this field (McHugh 2012) and are not aways included in the statistical evaluation exactly. Some researchers claim that a statistical calculation of agreement is not mandatory (e.g. Stelma & Cameron 2007). This presentation intends to provide more questions than answers, but at the same time provide suggestions to scientists tackling the questions of how to perform formal studies of human movements.publishersversionpublishe

A Formal and Functional Analysis of Gaze, Gestures, and Other Body Movements in a Contemporary Dance Improvisation Performance

UID/FIL/00183/2019 PTDC/FER‐FIL/28278/2017This study presents a microanalysis of what information performers “give” and “give off” to each other via their bodies during a contemporary dance improvisation. We compare what expert performers and non-performers (sufficiently trained to successfully perform) do with their bodies during a silent, multiparty improvisation exercise, in order to identify any differences and to provide insight into nonverbal communication in a less conventional setting. The coordinated collaboration of the participants (two groups of six) was examined in a frame-by-frame analysis focusing on all body movements, including gaze shifts as well as the formal and functional movement units produced in the head–face, upper-, and lower-body regions. The Methods section describes in detail the annotation process and inter-rater agreement. The results of this study indicate that expert performers during the improvisation are in “performance mode” and have embodied other social cognitive strategies and skills (e.g., endogenous orienting, gaze avoidance, greater motor control) that the non-performers do not have available. Expert performers avoid using intentional communication, relying on information to be inferentially communicated in order to coordinate collaboratively, with silence and stillness being construed as meaningful in that social practice and context. The information that expert performers produce is quantitatively less (i.e., producing fewer body movements) and qualitatively more inferential than intentional compared to a control group of non-performers, which affects the quality of the performance.publishersversionpublishe

The Role of Eye Gaze and Body Movements in Turn-Taking during a Contemporary Dance Improvisation

Abstract This paper intends to contribute to the multimodal turn-taking literature by presenting data collected in an improvisation session in the context of the performing arts and its qualiquantitative analysis, where the focus is on how gaze and the full body participate in the interaction. Five expert performers joined Portuguese contemporary choreographer, João Fiadeiro, in practicing his Real Time Composition Method during an improvisation session, which was recorded and annotated for this study. A micro-analysis of portions of the session was conducted using ELAN. We found that intersubjectivity was avoided during this performance, both in the performers' bodily movements and mutual gaze; we extrapolate that peripheral vision was chiefly deployed as a regulating strategy by these experts to coordinate turn-taking. A macro-analysis comparing the data with an analogous one obtained from NonPerformers provides the context for a discussion on multimodality and decision-making

a study on coordinating turns in a contemporary dance improvisation exercise

UID/LIN/03213/2013How does a group of people collaborate and take turns when no speaking is allowed? Unlike previous studies on turn-taking (e.g. Duncan 1972; Sacks, Schegloff & Jefferson 1974; Kendon 1967; Ochs et al. 1996), the context of this inquiry is linguistically independent. The present study intends to contribute to the literature by presenting data collected in a silent improvisation session in the context of the performing arts and its quali-quantitative analysis, where the focus is on how the body, rather than speech, participates in collaborative decision-making. Five expert performers and five non-performers, joined by choreographer Jo ̃ao Fiadeiro, were filmed separately during a contemporary dance exercise, the ”Real-Time Composition Game” (Fiadeiro 2007). The Game involves participants sitting around a table, and through means of selfselection, performing single actions at a time on a table using various objects to develop compositions and learn the nature of improvisation. A micro-analysis of portions of the session was conducted using ELAN (Lausberg & Sloetjes 2009). The annotation scheme codes for: a) directedness behavior (spatial location and orientation of the body, gaze points, object interaction); b) a formal description of movement units (MUs) of the various articulators; and c) a hermeneutic tier categorizing the functional-semiotic interpretation of the MUs (following a hierarchical taxonomy: self-focused, context-focused; communication-focused). The first two levels of annotation have an objective quality; the third level, based on the previous ones, describes raters’ subjective interpretation of the participants’ movements. Despite completing the task both collaboratively and creatively, the non-performer group reverted to those turn-taking strategies common in everyday social interactions, minus those involving the vocal modality (i.e. frequent gaze shifts and communicative body movements). In contrast, we found that intersubjectivity was actively avoided by the expert group, both in the performers’ bodily movements and mutual gaze, with turn management being regulated by means of alternative cognitive and social strategies, which will be presented. Besides the differences in communicative body movements across the groups, we will also compare self-focused movements, produced as neurophysiological responses to a cognitive load. A qualitative macro-analysis of the two groups’ entire sessions will focus on features directly related to the decision-making process throughout the improvisation exercises, such as hesitation versus determination. These differences will be analyzed under the light of recent literature focusing on social cognition and decision-making (inter alia Frith & Singer 2008). Constraints such as common knowledge, alignment, trust and the interaction of reason and emotion will be taken into account to contrast the results between the groups. The results of these analyses and their implications for computational modeling of turns in the context of of multimodality, as well as the relevance with questions of embodiment, creativity, and performance will be discussed together with future research.publishersversionpublishe

The ALICO Corpus: Analysing the Active Listener

Malisz Z, Wlodarczak M, Buschmeier H, Skubisz J, Kopp S, Wagner P. The ALICO Corpus: Analysing the Active Listener. Language Resources and Evaluation. 2016;50(2):411–442.The Active Listening Corpus (ALICO) is a multimodal data set of spontaneous dyadic conversations in German with diverse speech and gestural annotations of both dialogue partners. The annotations consist of short feedback expression transcriptions with corresponding communicative function interpretations as well as segmentations of interpausal units, words, rhythmic prominence intervals and vowel-to-vowel intervals. Additionally, ALICO contains head gesture annotations of both interlocutors. The corpus contributes to research on spontaneous human--human interaction, on functional relations between modalities, and timing variability in dialogue. It also provides data that differentiates between distracted and attentive listeners. We describe the main characteristics of the corpus and briefly present the most important results obtained from analyses in recent years

"Sinnepoppen" van Roemer Visscher – een realistische embleembundel uit de 17e eeuw in de collectie van de Universiteitsbibliotheek Wrocław (Polen)

  This article gives a detailed description of two prints of Roemer Visscher’s "Sinnepoppen": the first edition from 1614 and the later edition entitled "Zinne-Poppen" without year date elaborated and published by Visscher’s daughter Anna. Copies of both prints are to be found in the University Library in Wrocław among many other old prints from the Netherlands. The complicated history of this library as well as the history of Visscher’s work is given, and also a description and comparison of the two Wrocław copies, with their book history, provenances, today’s shape, remarks of the readers and emblematical structure.  This article gives a detailed description of two prints of Roemer Visscher’s "Sinnepoppen": the first edition from 1614 and the later edition entitled "Zinne-Poppen" without year date elaborated and published by Visscher’s daughter Anna. Copies of both prints are to be found in the University Library in Wrocław among many other old prints from the Netherlands. The complicated history of this library as well as the history of Visscher’s work is given, and also a description and comparison of the two Wrocław copies, with their book history, provenances, today’s shape, remarks of the readers and emblematical structure.  This article gives a detailed description of two prints of Roemer Visscher’s "Sinnepoppen": the first edition from 1614 and the later edition entitled "Zinne-Poppen" without year date elaborated and published by Visscher’s daughter Anna. Copies of both prints are to be found in the University Library in Wrocław among many other old prints from the Netherlands. The complicated history of this library as well as the history of Visscher’s work is given, and also a description and comparison of the two Wrocław copies, with their book history, provenances, today’s shape, remarks of the readers and emblematical structure

“Veracht ons Hollandt niet, wy hebben schooen Koeyen, Daer uyt dat soete-melck, en room, en boter vloeyen…”. De oude Nederlanden weerspiegeld op de blauwe tegels in Krzyżowice

The purpose of this paper is to show a collection of Dutch azure tiles stored in an old palace in Krzyżowice (Lower Silesia, Poland). The article presents a brief history of the palace and shows the images applied to decorate the tiles, focusing mainly on the image of a Dutch village: landscapes with bridges, mills, boats, scenic pieces presenting fishermen at work, people skating, animals (especially cows). The text is to show the way in which the tile images and other products of Dutch culture relate to one another (copperplate engravings, poetry). The paper thus seeks to present what makes this collection of tiles interesting in this particular part of Europe.The purpose of this paper is to show a collection of Dutch azure tiles stored in an old palace in Krzyżowice (Lower Silesia, Poland). The article presents a brief history of the palace and shows the images applied to decorate the tiles, focusing mainly on the image of a Dutch village: landscapes with bridges, mills, boats, scenic pieces presenting fishermen at work, people skating, animals (especially cows). The text is to show the way in which the tile images and other products of Dutch culture relate to one another (copperplate engravings, poetry). The paper thus seeks to present what makes this collection of tiles interesting in this particular part of Europe.The purpose of this paper is to show a collection of Dutch azure tiles stored in an old palace in Krzyżowice (Lower Silesia, Poland). The article presents a brief history of the palace and shows the images applied to decorate the tiles, focusing mainly on the image of a Dutch village: landscapes with bridges, mills, boats, scenic pieces presenting fishermen at work, people skating, animals (especially cows). The text is to show the way in which the tile images and other products of Dutch culture relate to one another (copperplate engravings, poetry). The paper thus seeks to present what makes this collection of tiles interesting in this particular part of Europe

Vechtend met woorden en melodie. Het lied als protestmiddel in Nederland

In scientific research in the Netherlands, there is not yet a publication fully dedicated to contemporary Dutch-language protest songs. Laurens Ham’s book Op de vuist. Vijftig jaar politiek en protestliedjes in Nederland, published in 2020, is therefore pioneering. It shows Dutch protest music and its relationship to the changing political and social situation in the Netherlands over a period of fifty years, that is, from the 1960s to the first two decades of the 21st century.In scientific research in the Netherlands, there is not yet a publication fully dedicated to contemporary Dutch-language protest songs. Laurens Ham’s book Op de vuist. Vijftig jaar politiek en protestliedjes in Nederland, published in 2020, is therefore pioneering. It shows Dutch protest music and its relationship to the changing political and social situation in the Netherlands over a period of fifty years, that is, from the 1960s to the first two decades of the 21st century.In scientific research in the Netherlands, there is not yet a publication fully dedicated to contemporary Dutch-language protest songs. Laurens Ham’s book Op de vuist. Vijftig jaar politiek en protestliedjes in Nederland, published in 2020, is therefore pioneering. It shows Dutch protest music and its relationship to the changing political and social situation in the Netherlands over a period of fifty years, that is, from the 1960s to the first two decades of the 21st century